A Unified View of Large-scale Zero-sum Equilibrium Computation

The task of computing approximate Nash equilibria in large zero-sum extensive-form games has received a tremendous amount of attention due mainly to the Annual Computer Poker Competition. Immediately after its inception, two competing and seemingly different approaches emerged---one an application of no-regret online learning, the other a sophisticated gradient method applied to a convex-concave saddle-point formulation. Since then, both approaches have grown in relative isolation with advancements on one side not effecting the other. In this paper, we rectify this by dissecting and, in a sense, unify the two views.Comment: AAAI Workshop on Computer Poker and Imperfect Informatio

Automatic assessment of sequence diagrams

In previous work we showed how student-produced entity-relationship diagrams (ERDs) could be automatically marked with good accuracy when compared with human markers. In this paper we report how effective the same techniques are when applied to syntactically similar UML sequence diagrams and discuss some issues that arise which did not occur with ERDs. We have found that, on a corpus of 100 student-drawn sequence diagrams, the automatic marking technique is more reliable that human markers. In addition, an analysis of this corpus revealed significant syntax errors in student-drawn sequence diagrams. We used the information obtained from the analysis to build a tool that not only detects syntax errors but also provides feedback in diagrammatic form. The tool has been extended to incorporate the automatic marker to provide a revision tool for learning how to model with sequence diagrams

Toward the automated assessment of entity-relationship diagrams

The need to interpret imprecise diagrams (those with malformed, missing or extraneous features) occurs in the automated assessment of diagrams. We outline our proposal for an architecture to enable the interpretation of imprecise diagrams. We discuss our preliminary work on an assessment tool, developed within this architecture, for automatically grading answers to a computer architecture examination question. Early indications are that performance is similar to that of human markers. We will be using Entity-Relationship Diagrams (ERDs) as the primary application area for our investigation of automated assessment. This paper will detail our reasons for choosing this area and outline the work ahead

Solving Large Extensive-Form Games with Strategy Constraints

Extensive-form games are a common model for multiagent interactions with imperfect information. In two-player zero-sum games, the typical solution concept is a Nash equilibrium over the unconstrained strategy set for each player. In many situations, however, we would like to constrain the set of possible strategies. For example, constraints are a natural way to model limited resources, risk mitigation, safety, consistency with past observations of behavior, or other secondary objectives for an agent. In small games, optimal strategies under linear constraints can be found by solving a linear program; however, state-of-the-art algorithms for solving large games cannot handle general constraints. In this work we introduce a generalized form of Counterfactual Regret Minimization that provably finds optimal strategies under any feasible set of convex constraints. We demonstrate the effectiveness of our algorithm for finding strategies that mitigate risk in security games, and for opponent modeling in poker games when given only partial observations of private information.Comment: Appeared in AAAI 201

Solving Games with Functional Regret Estimation

We propose a novel online learning method for minimizing regret in large extensive-form games. The approach learns a function approximator online to estimate the regret for choosing a particular action. A no-regret algorithm uses these estimates in place of the true regrets to define a sequence of policies. We prove the approach sound by providing a bound relating the quality of the function approximation and regret of the algorithm. A corollary being that the method is guaranteed to converge to a Nash equilibrium in self-play so long as the regrets are ultimately realizable by the function approximator. Our technique can be understood as a principled generalization of existing work on abstraction in large games; in our work, both the abstraction as well as the equilibrium are learned during self-play. We demonstrate empirically the method achieves higher quality strategies than state-of-the-art abstraction techniques given the same resources.Comment: AAAI Conference on Artificial Intelligence 201

Theoretical and Practical Advances on Smoothing for Extensive-Form Games

Sparse iterative methods, in particular first-order methods, are known to be among the most effective in solving large-scale two-player zero-sum extensive-form games. The convergence rates of these methods depend heavily on the properties of the distance-generating function that they are based on. We investigate the acceleration of first-order methods for solving extensive-form games through better design of the dilated entropy function---a class of distance-generating functions related to the domains associated with the extensive-form games. By introducing a new weighting scheme for the dilated entropy function, we develop the first distance-generating function for the strategy spaces of sequential games that has no dependence on the branching factor of the player. This result improves the convergence rate of several first-order methods by a factor of $\Omega(b^dd)$, where $b$ is the branching factor of the player, and $d$ is the depth of the game tree. Thus far, counterfactual regret minimization methods have been faster in practice, and more popular, than first-order methods despite their theoretically inferior convergence rates. Using our new weighting scheme and practical tuning we show that, for the first time, the excessive gap technique can be made faster than the fastest counterfactual regret minimization algorithm, CFR+, in practice

Experiments in the automatic marking of ER-Diagrams

In this paper we present an approach to the computer understanding of diagrams and show how it can be successfully applied to the automatic marking (grading) of student attempts at drawing entity-relationship (ER) diagrams. The automatic marker has been incorporated into a revision tool to enable students to practice diagramming and obtain feedback on their attempts

Using patterns in the automatic marking of ER-Diagrams

This paper illustrates how the notion of pattern can be used in the automatic analysis and synthesis of diagrams, applied particularly to the automatic marking of ER-diagrams. The paper describes how diagram patterns fit into a general framework for diagram interpretation and provides examples of how patterns can be exploited in other fields. Diagram patterns are defined and specified within the area of ER-diagrams. The paper also shows how patterns are being exploited in a revision tool for understanding ER-diagrams

The role of labels in the automatic assessment of graph-based diagrams

The ability to draw diagrams in free-form is rarely found in e-assessment systems. This paper examines one crucial area which needs to be well understood if automatic marking of diagrams is to be feasible: the analysis of labels. Graph-based diagrams include geometrical elements of differing shapes connected by lines to carry the semantics of the domain being modelled. Some diagrams also use the relative location of elements to express information. Labels play a central role in providing meaning both as identifiers, distinguishing between geometric elements, and the values of attributes of elements. Such information is central to the process of comparing two diagrams when marking student-drawn diagrams. Student-drawn diagrams likely to contain errors – they are imprecise. The aim, therefore, is, in the presence of imprecision, to match a student diagram with a model solution and award a mark which is close to that which an expert human would give. This paper explores the specific problem of determining the similarity of labels in diagrams which has been successfully employed in marking student diagrams in a formative environment